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ON  THE  COMPUTATION  OF  OPTIMAL  DESIGNS  FOR  CERTAIN  TIME  SERIES 
MODELS  WITH  APPLICATIONS  TO  OPTIMAL  QUANTILE  SELECTION 
FOR  LOCATION  OR  SCALE  PARAMETER  ESTIMATION 

Abbreviated  Title:  Computation  of  Optimal  Designs  in  Time  Series 

By 

Randall  L.  Eubank,  Patricia  L.  Smith  and  Philip  W.  Smith 
Southern  Methodist  University  and  Old  Dominion  University 

ABSTRACT 

:■  Using  the  results  of  Chow  (1978)  on  the  optimal  placement  of  knots 
in  the  approximation  of  functions  by  piecewise  polynomials ,  an  algorithm 

is  presented  for  the  computation  of  optimal  designs  for  certain  time 

^  - 

series  models  considered  by  Eubank,  Smith  and  Smith  (1981a,  1981b).  The 

J 

ideas  underlying  this  algorithm  form  a  unified  approach  to  the  computa¬ 
tion  of  optimal  spacings  for  the  sample  quantiles  used  in  the  asymptotically 
best  linear  unbiased  estimator  of  a  location  or  scale  parameter. 

1.  INTRODUCTION 

Consider  the  linear  regression  model  in  which  a  stochastic  process, 

Y,  is  observed  having  the  form 

Y(t)  -  8f(t)  +  X(t),  t  e  [0,1]  ,  (1.1) 

where  8  is  an  unknown  parameter,  f  is  a  known  regression  function  and  X(*) 
is  a  zero  mean  process  with  known  covariance  kernel,  R.  The  X  process  is 
assumed  to  admit  k-1  quadratic  mean  derivatives  at  each  point  t  e  [0,1]. 

When  the  Y  process  is  observed  over  all  of  [0,1],  the  reproducing 
kernel  Hilbert  space  (RKHS)  techniques  developed  by  Parzen  (1961a,  1961b) 
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may  be  used  Co  construct  a  linear  unbiased  estimator  of  the  parameter 
S.  We  will  denote  this  estimator  by  8.  For  finite  sampling  schemes 
the  regression  design  problem  has  been  considered  by  Sacks  and  Ylvisaker  (1966, 
1968,  1970),  Wahba  (1971,  1974),  and  Eubank,  Smith  and  Smith  (1981a,  1981b). 


In  the  context  of  model  (1.1),  a  regression  design  is  a  set  of  non-coincident 
points  in  [0,1].  The  problem  of  design  selection  is,  therefore,  one  of 
choosing  from  among  the  members  of  the  class  of  all  n+2  point  designs 


D  : 
n 


{(t0,t1,...,tn+1):0 


C0  <  C1  < 


<  t 


n+1 


i}. 


where  means  "is  defined  as". 

It  is  assumed  throughout  this  paper  that  it  is  possible  to  sample 

not  only  the  Y  process  but  its  derivatives  as  well.  Given  TeD,  one 

n 

can  then  consider  the  estimation  of  8  by  an  estimator  based  on  the 
observation  set 

Y.  T  "  (Y(1)(t):t  e  T,  i-0 . k-1}  . 

tC,  I 


In  particular,  generalized  least  squares  may  be  utilized  to  obtain  the 
best  linear  unbiased  estimator  (BLUE)  of  8  using  the  observations  Y  . 

K  ,  1 

This  estimator  will  be  denoted  by  6,  T. 

An  optimal  n-point  design  for  model  (1.1)  is  a  T*eD  which  satisfies 


V(8 


k,T*' 


inf  V(8 
TeD 


k,  T 


)  . 


n 

Problems  pertaining  to  the  existence  of  optimal  designs  can  be  handled 
as  in  Sacks  and  Ylvisaker  (1966).  Sufficient  conditions  for  the  uniqueness 
of  optimal  designs  have  been  given  by  Wahba  (1971)  and  Eubank,  Smith  and 
Smith  (1981a,  1981b)  for  certain  types  of  covariance  kernels. 

Unfortunately,  it  will  not  always  be  possible  to  sample  the  deriva¬ 
tives  of  the  Y  process.  However,  results  regarding  6.  _ 

K»  T 

in  this  event  since,  as  noted  by  Wahba  (1971), 

inf  V(8  )  <  inf  V(8  )  <  Inf  V(8  ) 

TeD  ,  L  TeD  ’  TeD  1 
nk  n  n 


are  still  useful 


where  8^  is  the  BLUE  of  8  obtained  without  the  use  of  derivative  information, 

i.e.,  the  generalized  least  squares  estimator  of  8  obtained  from  model  (1.1) 

using  the  observation  set  YT  =  {Y(t):t  e  T) .  It  should  also  be  noted  that, 

for  the  process  considered  here,  the  work  of  Barrow  and  Smith  (1978)  has 

the  consequence  that  optimal  designs  for  8,  _  are  asymptotically  optimal 

*C,T 

for  8t-  In  addition,  when  k  -  2  the  optimal  designs  for  8^  T  are,  under 

certain  conditions  on  f,  precisely  the  optimal  designs  for  6^,(c.f.  Theorem  2.3 

of  Eubank,  Smith  and  Smith  (1981a)  and  Theorem  2.2  of  Eubank,  Smith  and 

Smith  (1981b)).  Of  course  in  the  important  case  of  k“l  considered  by  Sacks 

and  Ylvisaker  (1966),  8.  _  ”  8_  and  our  work  is  also  applicable  to  the 

1 ,  T  T 

regression  design  problem  in  this  instance. 

In  this  paper  we  continue  the  work  of  Eubank,  Smith  and  Smith  (1981a, 
1981b)  by  constructing  an  algorithm  for  the  computation  of  optimal  designs 
for  the  case  that  R  is  the  covariance  kernel  corresponding  to  a  (k-l)-fold 
multiple  integral  of  a  Brownian  bridge  or  Brownian  motion  process  or  certain 
generalizations  of  these  processes.  The  case  k**l  corresponds  to  the 
Brownian  bridge  and  Brownian  motion  covariance  kernels  and  is  of  particular 
importance.  In  fact,  a  model  of  the  form  (1.1)  with  X(*)  a  Brownian  bridge 
process  has  been  shown  by  Parzen  (1979)  to  arise  in  the  estimation  of  a 
location  or  scale  parameter  by  linear  combinations  of  order  statistics. 

It  will  be  seen  (Section  4)  that  our  algorithm  can  be  used,  in  conjunction 
with  the  work  of  Eubank  (1981),  to  obtain  a  unified  framework  for  optimal 
spacing  selection  for  the  quantiles  utilized  in  the  asymptotically  best 
linear  unbiased  estimator  of  a  location  or  scale  parameter. 

In  Section  2  we  give  some  preliminary  results  regarding  certain  re¬ 
lationships  between  the  selection  of  designs  for  model  (1.1)  and  the 
approximation  of  functions  by  piecewise  polynomials.  Using  these  rela¬ 
tionships  it  is  possible  to  obtain  an  algorithm  for  optimal  design 
computation  through  the  modification  of  work  by  Chow  (1978)  on  piece- 


wise  polynomial  approximation  with  variable  knots.  The  optimal  design 
algorithm  is  presented  in  Section  3  along  with  several  illustrations 
of  its  use.  Its  application  to  location  or  scale  parameter  estimation 
is  discussed  in  Section  A.  Section  5  contains  a  short  summary. 

2.  OPTIMAL  DESIGNS  AND  PIECEWISE  POLYNOMIAL  APPROXIMATION 


The  covariance  kernel, R,  for  the  process  (1.1)  is  the  reproducing 
kernel  for  a  reproducing  kernel  Hilbert  space  (RKHS)  which  will  be  denoted 
as  H(R)  (c.f.  Parzen  (1961a,  1961b).  The  problem  of  optimal  design  selection 


for  the  estimator  6.  T  may  be  formulated  as  a  minimum  norm  approximation 

k,  T 

problem  in  H(R)  in  the  following  manner.  Let  ||*||  denote  the  norm  in 

R 

H(R)  and  define 

T:  -  span{R^°’^(*,t)  :  t  e  T,j  -  0,l,...,k-l} 


where 

R(i’^: 


31+j 

r  R(8,0 
3s  3tJ 


Thus  we  see  that  the  optimal  design  problem  is  equivalent  to  the  non¬ 
linear  best  approximation  problem:  Find  T*eD  such  that  s*:«P,  „*f 

n  k,  l7* 
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satisfies 


| | f-s* | L  -  inf  inf  |  |  f-s  |  |  . 

K  TeD  seS,  _ 
n  k,T 

In  order  to  study  this  problem  more  closely  we  now  restrict  our 
attention  to  a  specific  class  of  X  processes  and  their  corresponding 
covariance  kernels .  Let 

1  .  .k-1 ,  ,k-l 

K(s,t ):*  /  (s-u),+  du  t  o  <  a,  t  <  1,  (2.4) 

0  (k-l)r 

r  r 

where  (x)+  »  x  for  x  _>  0  and  is  0  otherwise,  and  let  U(’)  denote 
the  corresponding  normal  process,  i.e.,  a(k-l)-fold  multiple  integral 
of  Brownian  motion.  Define  a  new  process,  W,  by 


W(t) 


U(t)-E[U(t)|U(j)(l),  j-k-q . k-1] ,  0  <  q  _<  k  , 


U(t) 


(2.5) 


It  will  be  assumed  in  subsequent  discussions  that  R  is  the  covariance 
kernel  defined  by 

R(s,t):  ■  Cov(W(s) ,  W(t)).  (2.6) 

When  q  -  1,  R  is  the  covariance  kernel  corresponding  to  a  (k-l)-fold 
multiple  integral  of  a  Brownian  bridge  process  whereas  the  case  of  q-0 
reverts  to  the  covariance  kernel  (2.4)  for  a  multiple  integral  of  Brownian 
motion.  The  case  of  q«k  was  considered  by  Eubank,  Smith  and  Smith  (1981a). 

For  processes  with  covariance  kernels  of  the  form  (2.6),  it  is  known 
that  (c.f.  Eubank,  Smith  and  Smith  (1981a,  1981b)), 

i)  H(R)  is  a  Hilbert  function  space  consisting  of  functions  which 
satisfy  for  feH(R), 

f*'"^  absolutely  continuous,  j“0 . k-1,  with  f^c  L^[0,1], 

and  boundary  conditions 

f(j)(0)-0,  j«0, . . . ,k-l 

f^(l)-0,  j-k-q,..., k-1,  for  0  <  q  <_  k  , 


or  just  f^(0)  *  0,  j«0, . . .  ,k-l,  for  q  "  0.  The  norm  for  feH(R)  is 


I  ML  -  lif(k)||  ,  -  {/1(f(k)(x))2dx}1/2  .  (2.7) 

R  L2  0 


ii)  When  q  «  k,  observations  at  0  or  1  provide  no  information 
regarding  the  parameter  6  (this  follows  from  property  i)  since,  in  this  case, 
f(j)(0)  »  f^(l)  -  0,  j  ■  0,...,k-l).  Therefore,  only  design  points 
in  the  interior  of  [0,1]  are  utilized  when  estimating  8.  This  convention 
has  the  consequence  that  the  designs  to  be  selected  here  agree  with  those  con¬ 
sidered  by  Eubank,  Smith  and  Smith  (1981a).  We  note  in  passing  that 
similar  remarks  about  observation  selection  hold  for  other  values  of  q 
(c.f.  Sacks  and  Ylvisaker  (1966)  for  the  case  of  k-1,  q»0) . 


iii)  R(s,t),  as  a  function  of  s  for  fixed  t,  is  a  spline  of  order  2k 


in  continuity  class  C  with  a  knot  at  t. 

2  (k) 

iv)  For  T-(tn,t. , . . . ,t  .,)eD  ,  the  best  L  [0,1]  approximation  to  f 
u  l  n-ri  n 

from  P  (T) ,  the  set  of  piecewise  polynomials  of  order  k  with  breakpoints 
at  is  (\jTf)( 

2  k 

Let  T  denote  the  L  [0,1]  orthogonal  projector  for  P  (T) .  Then, 

equation  (2.7)  and  result  iv)  have  the  consequence  that 


I  f  "  Pk,Tf ' 'r 


and,  hence. 


inf  I  1  f-P,  J 


k,T  1 'R 


inf | |f 00 -Q^  Tf(k)||  2  . 

TeD  ’  L 

n 


Therefore,  in  view  of  (2.3),  the  optimal  design  problem  for  the  types 

of  Y  processes  considered  here  coincides  with  finding  the  breakpoints 

2  (k) 

of  the  best  L  [0,1]  piecewise  polynomial  approximation  to  f  . 

Approximation  by  piecewise  polynomials  with  free  breakpoints  has 

been  studied  by  Barrow,  et  al  (1978),  Barrow  and  Smith  (1978),  and  Chow 


(1978).  Their  results,  restated  in  the  design  setting,  yield  this 
partial  characterization  of  optimal  designs  for  covariance  kernels 
of  the  form  (2.6). 

Theorem  1.  Let  T*  *  (t*, t* . t*. . )  e  D  be  an  optimal  design.  If 

-  u  l  n+i  n 

f  e  H(R)OC2k[0,l]  and  f(2k)>  0  on  [0,1]  then,  for  i  -  l,...,n, 

(\  T*f^k^ti+^  »  k  even 

(2.8 

2f°°(t*)  -  (pk>T*f)(k>(t*+),  k  odd  . 

Sometimes  the  necessary  condition  (2.8)  is  also  sufficient  to 
guarantee  an  optimal  design.  We  state  such  a  result  from  Eubank,  Smith 
and  Smith  (1981a,  1981b). 

Theorem  2.  Let  f  e  H(R)  O  C2k  [0,1]  with  f(2k)  >  0  on  [0,1]  and  log  f(2k 
concave  on  (0,1),  then  8^  has  a  unique  optimal  design  for  each  n. 

In  general  uniqueness  is  quite  difficult  to  prove.  At  present,  very 
few  other  positive  results  concerning  uniqueness  are  available. 


(P  f)^k\t*-) 
^  k,T*  '  i  ' 


3.  AN  ALGORITHM  FOR  COMPUTING  OPTIMAL  DESIGNS 


Theorem  1  suggests  that  we  should  find  a  design  T  *  (tn, . ,.,t  .,) 

u  n+i 

for  which  F^(T)  ■  0,  i  *  l,...,n,  where 

,  (k) , ^  /D  £.  (k) 


Ft(T) 


(  (Pk,Tf)  (ti+)  -  <Pk,Tf)  ^ 

(  2f Ck)  (t±)  -  (pkjTf)(k)(tr)  -  (Pk>Tf)(k)(ti+), 


k  even 
k  odd. 


(3 .  J 


Thus,  setting  F(T)  -  (F^T) 


,Fn(T)) 


we  see  that  we  are  looking  for  a 


zero  of  the  vector  valued  function  F.  Such  zeros  will  be  candidates  for 
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optimal  designs.  Chow  (1978)  has  shown  that 


Fi(T) 


(ti  "  ti-l)  rl„  ,  k-1  k„(2k)  . .  . 

-  j  (1-t)  t  fv  '[t  .+  x(t  -t  )]di 

(k-1) !  0  i-i  i  i-l 


(3.2) 


(ti+rti)  rl  „  vk  k-1 ^( 2k)  r .  . 

■  . V  T  f  [ti+r(ti+rt1)JdT. 


i  -  1, .. . ,n, 


where, recall  tg:=0  and  tn+^:»l.  Consequently,  the  Jacobian  matrix  of  F  at  T, 


3F 


is  tridiagonal  with  non-zero  elements  given  by 


k(ti-ti-l)  rl,.  ,k  k-l,(2k)r.  . 

- -  /  (1-t)  t  f  [t^H-  x  (ti-ti_1)]dx,  (3.3) 

2  £  i  £  n  , 


(t  -t  )*"-*■ 

-  “Tkiiyf - /1(i-T)k'2xk(kx-i)f(2k)[t._1+x(ti-t1_1)]dx 

+  ~Tk-l)! -  /1(l-T)kxk“2(k(l-x)-l)f(2k)[t!i+x(t.+1-ti)]dx,  (3.4) 

1  <  i  <  n  , 


k(ti+rti)  r1, 


/'(l-t)k”1Tkf(2k)[t1  +  T(t1+1-t1)] 


(k-1) ! 


(3.5) 


1  <  i  <  n  -  1  . 


When  f  is  2k  times  continuously  differentiable  and  fv  >  0  we 

can  use  Newton's  method  to  find  a  T*eD  which  is  a  zero  of  F.  Such  a 

n 

T*  will  be  an  optimal  design  candidate  and  may  be  constructed  using  the 
algorithm  presented  below.  If,  in  addition,  f  satisfied  the  conditions 
of  Theorem  2,  then  the  T*  located  by  the  algorithm  will  be  the  optimal 
design. 
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Algorithm: 

Step  1.  Select  an  initial  T  =  (t^, . . . »tn+^) • 

Step  2.  Check  to  insure  that  TeD  . 

n 

Step  3.  Compute  F(T)  and  A(T) . 

Step  4.  Compute  b  =  A(T)  ^F(T) . 

Step  5.  Stop  if  b  is  small  or  the  maximum  number  of  iterations 
has  been  met. 

Step  6.  Set  t^  =  t^  -  b^,  i=l,...,n,  and  return  to  Step  2. 


As  was  indicated  above,  the  algorithm  (when  it  converges)  finds  a 
design  T*  which  satisfies  a  necessary  condition  for  design  optimality. 

In  order  to  enhance  our  chances  of  finding  a  "good"  design, care  should 
be  taken  in  Step  1.  An  initial  design  choice  which  usually  yields  good 
results  is  the  n^  element  of  an  asymptotically  optimal  design  sequence 
(c.f.  Sacks  and  Ylvisaker  (1966)).  Such  a  sequence  can  be  constructed 
using  the  density 
h(x)  = 

in  the  following  manner.  Let  H  denote  the  distribution  function  corre¬ 
sponding  to  h  with  associated  inverse  (or  quantile)  function  H  .  Then 
it  can  be  shown  (c.f.  Eubank,  Smith  and  Smith  (1981a,  1981b)  and/or  Sacks 
and  Ylvisaker  (1966,  1970))  that  the  nth  element  of  an  asymptotically 
optimal  design  sequence  for  S,  _  consists  of  the  points 

K  j  I 

ti-H_1(^~),  i«0,...,n+l.  (3.7) 

00 

The  design  sequence  {T  }  ,  obtained  by  solving  (3.7)  for  successive 

n  n»l 

values  of  n  is  asymptotically  optimal  in  the  sense  that 

V(8,  T  )-V(8) 

n _ 

inf  V(6r  T)  -  V(S) 

TeD  ' 
n 


|f(2k)(x),2/2k+1/  /1lf(2k)(s)|2/2k+lds> 
0 


(3.6) 


lim 

n 


■  1  . 


(3.8) 


10 


Although  this  relationship  between  optimal  and  asymptotically  optimal 
designs  pertains  to  large  n,  it  is  often  the  case  (as  will  be  discussed 
in  the  examples)  that  the  asymptotics  carry  over  to  small  n  at  least  to 
the  extent  that  the  values  of  asymptotically  optimal  designs  provide  a 
good  indication  of  the  locations  of  the  optimal  design  points. 

In  some  cases  H  ^  has  a  closed  form  making  asymptotically  optimal 
designs  easy  to  compute.  However,  even  when  this  is  not  the  case,  H  ^ 
can  be  readily  evaluated  through  numeric  tabulation  of  H  and  subsequent 
interpolation. 

If  after  one  or  more  iterations  the  check  in  Step  2  fails,  this 
indicates  that  the  algorithm  has  moved  out  of  the  feasible  region,  D^. 

Such  an  occurance  is  usually  indicative  of  a  poor  choice  for  an  initial 
design.  In  this  event  one  alternative,  of  course,  is  to  simply  reinitialize 
with  another  design  and  try  again.  Alternatively,  one  might  reduce  the  size 
of  the  step  taken  in  Step  6,  i.e.,  take 

t .  =  t .  -  ab .  (3.9) 

ill 

for  some  0<a<l.  More  generally,  a  modified  version  of  the  algorithm  could 
be  utilized,  where  for  instance,  after  the  ith  iteration  the  new  design 
points  are  taken  as 

-  oSjbj,  j  =  1, . . . »n,  (3.10) 

for  some  0<a<l  where  0<S^<L  is  the  largest  value  such  that  the  resulting  design 

remains  in  D  .  We  have  not  tested  this  last  modification  since  it  would  only 
n 

reduce  convergence  time  and  since,  for  all  the  problems  we  have  considered, 
convergence  has  always  occurred  (in  terms  of  a  relative  change  in  the  design 
points  between  successive  interations  of  less  than  10  ^)  after  only  4  to 
7  interations  using  an  asymptotically  optimal  initial  design. 

The  integrals  computed  in  Step  3  of  the  algorithm  will  usually  require 
evaluation  by  numerical  methods.  This  can  be  readily  accomplished  through 
the  use  of  a  Gaussian  quadrature  rule. 
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We  conclude  this  section  by  presenting  several  examples  which 
illustrate  the  use  of  the  algorithm  and  Theorems  1  and  2  in  the 
computation  of  optimal  designs.  For  simplicity  the  X  process  implicit 
in  each  of  the  following  examples  is  taken  to  have  covariance  kernel 
(2.4).  When  exhibiting  a  particular  design  we  present  only  those 
values  which  are  in  the  interior  of  [0,1]. 

Example  1.  Consider  first  the  case  of 

f(t)  -  t6  .  (3.11) 

This  regression  function  furnishes  an  example  of  a  function  which 
satisfies  the  conditions  of  Theorems  1  and  2  and,  hence,  in  this  case 
for  k  =1,2,3  we  are  assured  of  unique  optimal  designs  for  all  values  of 
n.  In  addition,  through  the  use  of  this  function  it  will  be  possible 
to  examine  instances  when  the  use  of  uniformly  spaced  design  points, 
either  for  estimation  or  for  initialization  of  the  algorithm,  is  a 
sound  strategy  as  well  as  cases  when  it  is  not.  We  now  consider  the 
construction  and  properties  of  various  designs  for  this  regression 
model . 


In  computing  optimal  designs  for  this  model  asymptotically  optimal 
designs  were  utilized  as  starting  values.  For  a  regression  function 


having  the  form  (3.11)  the  asymptotically  optimal  designs  are  simple  to 
compute  since  the  H  ^  function  is  given  by 


H_1(x)= 


when  k  =  1, 
when  k  =  2, 
when  k  ■  3  . 


It  is  important  to  note  that  for  k  =  3  the  asymptotically  optimal  designs 
consists  of  uniformly  spaced  design  points.  These  are,  in  fact,  seen  to 
be  the  optimal  designs.  Therefore,  in  the  case  of  f(t)  «  t^  with 
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k  «  3  sampling  the  Y  process  at  uniform  intervals  is  not  only  sound  but  an 
optimal  strategy.  This  is  not  the  case,  however,  for  k  ■  1  and  2. 

The  optimal  designs  of  size  n  -  1,3,5,10,20  for  this  regression  model 
were  computed  for  k  »  1,2,3.  The  variances  of  6,  T  corresponding  to 
these  designs  are  presented  in  Table  1  along  with,  for  comparison 
purposes,  the  variances  obtained  through  the  use  of  asymptotically 
optimal  and  uniform  designs.  These  values  may  be  compared,  from  a 
regret  point  of  view,  with  the  values  of  V(0)  ■  ||f^||  ^  provided  at 
the  bottom  of  the  table. 

Examination  of  Table  1  reveals  that,  as  one  might  suspect,  the  sub¬ 
stantive  gains  from  the  use  of  optimal  (as  opposed  to  asymptotically 
optimal)  designs  occur  for  small  n  and/or  k.  Asymptotically  optimal 
designs  perform  quite  well  in  this  case  even  for  relatively  small  n  over 
all  values  of  k.  In  contrast,  for  k  *  1  uniformly  spaced  design  points 
tend  to  perform  poorly,  relative  to  optimal  or  asymptotically  optimal 
designs,  especially  for  small  n.  The  use  of  uniform  designs  would  seem 
acceptable  for  large  n  when  k  »  2  and,  of  course,  the  uniform,  asymp¬ 
totically  optimal  and  optimal  designs  all  agree  when  k  *  3. 

Through  the  use  of  asymptotically  optimal  designs  to  initialize 
the  algorithm  it  was  possible  to  obtain  convergence  to  the  optimal  designs, 
in  every  instance,  in  5  or  fewer  iterations.  However,  when  uniformly 
spaced  design  points  were  tried  as  starting  values,  this  resulted  in  a 
failure  of  the  check  in  Step  2  of  the  algorithm  for  n  _>  10  when  k  »  2 
and  even  for  n  as  small  as  3  when  k  ■  1.  In  the  instance  of  k  *  2  it 
was  found  that  by  taking  a  step  of  size  5  ■  .29  in  (3.9)  convergence  (to 
the  extent  of  5  digit  accuracy)  could  be  obtained  after  41  iterations 
(values  of  5  >  .3  were  all  apparently  to  large  to  provide  similar  results). 
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No  such  value  of  6  could  be  found  when  k  «  1.  Keeping  in  mind  the 
criterion  (2.8)  that  is  utilized  for  locating  an  "optimal"  design 
point,  this  latter  fact  comes  as  no  surprise  when  one  compares,  for 
instance,  the  uniform  3  point  design  (.25,  .5,  .75)  with  the  optimal 
design  (.65828,  .81674,  .92042)  for  this  case. 


Example  2.  As  an  example  of  a  regression  function  which  is  not  a 
polynomial  we  now  suppose  f  has  the  form 

««  -  its  ‘7/2  • 


where  the  factor  8/105  is  introduced  to  simplify  subsequent  numerical 

presentations.  Both  Theorems  1  and  2  are  applicable  to  this  function 

(2k) 

when  k  =  1  but  not  when  k  *  2  or  3  as,  in  these  latter  cases,  f  is 
not  in  C[0,1].  Consequently,  this  will  provide  an  illustration  of  the 
performance  of  the  algorithm  under  conditions  other  than  those  of 
Theorem  1  or  the  ideal  conditions  of  Theorem  2. 

As  in  the  previous  example  the  H  ^  function  has  a  closed  form.  In 
this  case 


H_1(x) 


X1/2 

when 

?? 

11 

X5/A 

when 

k  - 

x7/2 

when 

k  - 

The  variances  of  8,  _  corresponding  to  optimal,  asymptotically  optimal 
K,  T 

and  uniformly  spaced  designs  of  size  n-1,3,5,10,20  are  presented  in  Table  2 
for  k  ■  1,2,3.  As  in  the  previous  example,  optimal  designs  were  computed 
using  asymptotically  optimal  starting  values.  In  all  instances  convergence 
occurred  after  at  most  7  iterations.  Examinations  of  the  values  in  Table  1 
lead,  again,  to  the  conclusion  that  the  use  of  optimal  designs  (rather 
than  asymptotically  optimal  designs)  will  be  of  the  most  value  when 
n  is  small.  It  should  be  remembered,  however,  that  in  contrast  to 


TABLE  2.  VARIANCE  OF  T  FOR  VARIOUS  DESIGNS  WHEN  f(t)  =  jqJ 
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Che  results  presented  in  Table  1,  those  given  in  Table  2  were  obtained, 
for  k  *  2,3,  when  Theorem  1  is  not  applicable.  This  suggests  that  the 
algorithm  may  still  perform  well  under  moderate  departures  from  the 
assumptions  upon  which  it  is  based. 

Example  3.  Finally  consider  the  case  of  f  having  the  form 

f(t)  -  (t  -  -|)8  -  (±)8  +  8(|)7t.  (3-12) 

This  regression  function  provides  an  example  of  optimal  design  duplicity 

as  well  as  an  illustration  of  the  sensitivity  some  functions  exhibit 

regarding  the  selection  of  an  initial  design. 

Using  n“l  with  k*2  one  finds  that  V(80  _)  has  a  local  maximum 

T 

(rather  than  minimum)  at  T°-{.5}  with  V(82  T<»)  ■  .74681.  There  are 

two  optimal  designs  T^  ■  {.23079}  and  T^  *  {.76921}  where  V(8„  1)  » 

Z  *T 

V(S2  t2)  -  .72007.  Choices  of  starting  values  such  as  .6  or  .4  lead 

to  convergence  to  T°  whereas,  for  instance,  the  choices  .2  and  .8 

1  2 

results  in  convergence  to  T  and  T  respectively.  It  should  be  noted 
that  T°  is  also  the  uniform  and  asymptotically  optimal  design  for  this 
case.  We  therefore  have  an  instance  when  the  use  of  either  uniform  or 
asymptotically  optimal,  in  lieu  of  optimal,  designs  is  not  only  poor, 
but  in  fact,  the  worst  strategy. 

The  function  (3.12)  has  also  been  considered  in  Bock  (1976)  where 
the  graph  of  | | f-P2  Ir  versus  t^  is  seen  to  have  a  "W-shape".  Although 
there  is  little  difference,  in  this  case,  between  the  variance  at  the 
local  maximum  and  at  the  two  minimums,  it  is  clear  that  functions  may  be 
constructed  for  which  this  difference  is  arbitrarily  large. 

In  the  examples  we  have  considered  only  regression  functions  for 
which  the  H  ^  function  has  a  closed  form.  This  has  been  for  the  sake 


I 

I 
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of  Illustration  and  has  the  consequence  that  the  asymptotically  optimal 
designs  perform  better,  for  estimation  purposes,  than  might  otherwise 
be  the  case.  Unfortunately,  situations  where  H  ^  is  of  a  closed  form 
are  rare  in  practice.  The  reader  is  refered  to  Eubank  (1979)  for 
several  examples  of  the  evaluation  of  H  ^  by  numerical  methods. 

All  the  computations  in  this  and  subsequent  sections  were  per¬ 
formed  on  either  the  IBM  360  computer  at  Arizona  State  University  or 
the  CDC  6600  at  Southern  Methodist  University. 


4.  APPLICATION  TO  LOCATION  OR  SCALE  PARAMETER  ESTIMATION 


Suppose  a  random  sample,  Z, , ...,Z„  is  obtained  from  a  distribution 

1  N 

of  the  form  F(z)  «  F^Cz/B),  where  F^  is  a  known  distributional  form  and 
B  is  an  unknown  scale  parameter.  F^  is  assumed  to  be  absolutely  continuous 
with  associated  probability  density  function  f^.  Let  QQ(t) :«Fp1(t)  and 
define  the  density-quantile  function  as  dQ(t) :»f0(Q0(t)) ,  0  <  t  <  1.  The 
sample  quantile  function  is  defined  by  Q(t)  -  Z^,  <  t  £  ^  ,  j-l,...,N 

where  Z^  denotes  the  sample  order  statistic. 

Parzen  (1979)  has  shown  that,  for  N  sufficiently  large,  a  model  for 
scale  parameter  estimation  is 


dQ(t)Q(t)  -  8d0(t)Q0(t)  +  aX( t) ,  t  e  [0,1],  (4.1) 

where  a  ■  S/i/tT  and  X(‘)  is  a  Brownian  bridge  process.  Eubank  (1981)  has 
shown  that  the  problem  of  optimal  design  selection  for  model  (4.1)  is 
identical  to  the  problem  of  selecting  an  optimal  spacing  for  the  sample 
quantiles  utilized  in  constructing  the  asymptotically  best  linear  unbiased 
estimator  (ABLUE)  of  B(c.f.  Sarhan  and  Greenberg  (1962)  for  discussions 
and  examples  of  the  more  classical  approach  to  the  optimal  spacing  problem) 


Given  a  design  TeD^  (classically  refered  to  as  a  spacing  in  the  con¬ 
text  of  this  problem)  the  ABLUE  of  8,8^,  is,  in  fact,  the  corresponding 
generalized  least  squares  estimator  of  B  formed  from  model  (4.1).  Since, 
for  a  Brownian  bridge  process,  k“q,  it  follows  from  property  ii)  in  Section  2 

that  only  those  observations  which  correspond  to  the  design  points, t, ,..., t  , 

1  n 

in  the  interior  of  [0,1]  are  used  for  estimation.  Thus,  8 T  is  of  the  form 
n 

£  c(ti)dg(ti)Q(ti)  where  explicit  expressions  for  the  c(t^)  can  be  found 
in  Sarhan  and  Greenberg  (1962) .  The  optimal  spacing  problem  consists 
of  finding  a  spacing  (design)  for  which  the  variance  of  B^,  is  a  minimum 
or,  equivalently, 

ARE(8t)  -  V(B)/V(Bt),  (4.2) 

the  relative  efficiency  of  8^  with  respect  to  B,  is  a  maximum.  Equation 
(4.2)  can  be  shown  to  provide  the  asymptotic  (as  N  -*■  «>)  relative  efficiency 
of  the  ABLUE  with  respect  to  the  maximum  likelihood  estimator  of  8  (c.f. 
Eubank  (1981))  which  indicates  the  reason  underlying  the  use  of  the  ARE 
notation. 

Upon  examination  of  (4.1)  there  may  appear  to  be  a  disparity  between 

this  model  and  the  regression  model  (1.1)  (and,  consequently,  between  the 

optimal  spacing  and  optimal  design  problems)  due  to  the  factor  a  which 

appears  in  (4.1).  The  presence  of  this  term  implies  that  the  variance 

of  8X  is  not  I  lPi,td0^  IfT  38  in  but  rathft-  °2|lr’i  Tdo^R  ‘  However> 

2 

a  ,  although  unknown,  is  independent  of  T.  Consequently,  to  minimize 

*  2 
the  variance  of  8T  it  suffices  to  minimize  | | TdQ| |~  and  the  optimal 

spacing  problem  is  therefore  equivalent  to  the  optimal  design  problem 
discussed  in  the  previous  two  sections. 

At  present  the  literature  on  optimal  spacings,  is  composed  of 
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numerous  articles  cataloging  the  optimal  spacings  for  various  distribu¬ 
tion  types  (c.f.  Eubank  (1981))  for  a  list  of  references).  Thus  the 
classical  approach  to  the  optimal  spacing  problem  has  been  to  consider 
each  distribution  separately.  As  the  Brownian  bridge  process  corresponds 
to  the  special  case  of  k  and  q  both  equal  to  1  it  now  follows  that  the 
algorithm  presented  in  the  previous  section  may  also  be  used  for  the 
computation  of  optimal  spacings.  Two  important  consequences  of  this 
fact  are: 

(i)  model  (4.1)  in  conjunction  with  Theorems  1  and  2  and  the 
algorithm  of  Section  3  provide  the  first  simple,  unified 
framework  for  the  computation  of  optimal  spacings. 

(ii)  through  reference  to  the  optimal  spacing  literature,  comparisons 
may  be  made  between  designs  (spacings)  obtained  from  the 
algorithm  and  those  computed  by  other  authors  using  the  classical 
approach  which  involves  a  search  using  global  optimization  for 
each  distribution. 

It  is  important  to  note  that  due  to  the  particular  characteristics  of  a 
distribution  it  is  sometimes  possible  to  show  uniqueness  for  optimal 
spacings  when  Theorem  2  is  not  applicable.  Such  results  may  be  helpful 
in  providing  an  indication  of  how  our  algorithm  will  perform  under  non¬ 
ideal  conditions.  We  illustrate  this  and  the  other  comments  with  an 
example. 

Let  Fq  be  the  distribution  function  for  the  Pareto  distribution, 

i .  e . , 

Fq(x)  -  1  -  (1  +  x)  V,  x,  v  >  0. 

In  this  case 

d0(t)qQ(t)  -  v[(l-t)  -  (1-t)  V  ] 


(4.3) 


TABLE  3.  OPTIMAL  SPACINGS  DESIGNS  FOR  THE  PARETO,  v 


-  .5,2,  k  -  1 


n 

1 

3 

7 

v  -  .5 

v  -  2 

\>  -  .5 

v  -  2 

v  -  .5 

v  -  2 

ci 

.35961 

.61809 

.16295 

.34049 

.07805 

.17865 

C2 

.35048 

.63042 

.16078 

.34519 

C3 

.58405 

.85886 

.24934 

.49868 

C4 

.34549 

.63789 

C5 

.45215 

.76118 

c6 

.57488 

.86617 

C7 

.72776 

.94889 

ARE(8X) 

.77461 

.72136 

.94657 

.92597 

.98700 

. 98088 
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and 

1  — 

[d0(t)Q0(t)]"  -  -(1  +  i)(l-t>  v  .  (4.4) 

Theorem  2  Is  therefore  applicable  and  insures  a  unique  optimal  design 
when  v  _<  1.  The  unique  optimal  spacings  for  v  =  .5  obtained  from  the 
algorithm  with  n  *  1,3,7  are  presented  in  Table  3  and  agree  with  those 
obtained  by  Kulldorf  and  Vannman  (1973)  using  global  optimization  methods. 
Also  given  in  Table  3  are  the  results  obtained  from  the  algorithm  when 
v  *  2.  Even  though  neither  Theorem  1  nor  2  applies  it  is  still  true 
(c.f.  Kulldorf  and  Vannman  (1973))  that  the  optimal  spacings  for  the 
Pareto  are  unique  in  this  case  as  well.  The  fact  that  the  spacings 
computed  by  the  algorithm  agree  with  the  optimal  spacings  for  v  «  2 
given  by  Kulldorf  and  Vannman  (1973)  is  an  important  illustration  of 
the  fact  that  unique  optimal  designs  exist  for  a  wider  class  of  functions 
than  those  satisfying  the  hypotheses  of  Theorems  1  and  2  and,  in  such 
instances,  may  be  computed  with  this  algorithm. 

If,  instead  of  scale  parameter  estimation,  location  parameter 
estimation  is  of  interest,  the  distribution  function  has  the  form 
F(z)  *  Fq(z  “  ^  model  similar  to  (4.1)  holds  in  this  case  as  well. 

To  obtain  an  algorithm  for  optimal  spacing  computation  in  this  instance 
it  is  only  necessary  to  interchange  the  roles  of  d^  and  dg*Qg  in  the 

previous  discussion. 

The  example  presented  in  this  section  illustrates  how  the  algorithm 
presented  in  Section  3  may  be  used  for  optimal  spacing  computation,  and, 
in  addition,  provides  an  indication  of  how  it  performs  under  departures 
from  the  "ideal  conditions"  of  Theorems  1  or  2.  For  these  reasons  it 
has  been  useful  to  consider  a  situation  where  the  optimal  spacings  had 
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been  obtained  by  other  methods  and  were,  therefore,  available  for  com¬ 
parison  purposes.  However,  the  value  of  this  algorithm  to  the  practitioner 
will  lie  in  its  use  for  the  computation  of  the  optimal  spacings  in  situations 
which  have  not  been  considered  in  the  literature  and  for  which  existing 
results  are  not  available.  It  is  our  belief,  based  on  comparison  with 
the  classical  results,  that  this  algorithm  will  be  a  valuable  tool  for 
this  purpose  even  under  moderate  departures  from  the  conditions  of 
Theorems  1  or  2.  We  also  conjecture  that,  since  the  algorithm  is  based 
on  the  local  behaviour  of  the  d(or  d*Q)  function  near  an  optimal  spacing 
element,  optimal  spacings  can  be  obtained  more  rapidly  and  efficiently 
through  the  use  of  this  method  rather  than  an  ad  hoc  global  optimization 
technique.  Unfortunately,  the  computational  aspects  of  optimal  spacing 
construction  are  typically  not  reported  in  the  literature  on  the  subject 
and  consequently,  it  is  difficult  to  obtain  comparisons  which  support 
this  contention. 

5.  SUMMARY  AND  DISCUSSION 

In  this  paper  an  algorithm  has  been  presented  for  the  computation  of 
optimal  designs  for  certain  time  series  models.  This  algorithm  locates 

a  design  which  satisfies  a  necessary  condition  for  optimality  provided 

(2k)  (2k) 

f  is  continuous  and  of  one  sign  on  [0,1].  If,  in  addition,  log  f 

is  concave  on  (0,1)  the  use  of  this  algorithm  should  provide  the  optimal 
design.  The  algorithm  has  also  been  shown  to  be  useful  in  the  selection 
of  order  statistics  for  location  or  scale  parameter  estimation.  The 
advantage  of  this  approach  to  spacing  selection  over  classical  techniques 
is  that  it  provides  a  unified  approach  to  optimal  spacing  selection  which 
obviates  the  need  for  global  optimization. 
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Experience  with  this  algorithm  indicates  that  it  works  rather  well 
even  when  the  conditions  of  Theorem  1  are  only  approximately  satisfied 
(e.g..  Example  2  of  Section  3  and  the  case  of  v  *  2  in  Section  4).  However, 
it  may  be  more  sensitive  in  such  cases  to  the  choice  of  initial  designs. 

While  uniformly  spaced  starting  values  are  easily  input  and  may  produce 
the  optimal  design,  they  can  also  give  poor  or  misleading  results.  Generally, 
better  results  may  be  obtained  by  initializing  with  an  asymptotically  optimal 
design,  and,  consequently,  this  method  is  recommended  even  though  one  must 
begin  by  evaluating  the  function  H  ^  as  in  (3.7). 
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